Python Generators

In this tutorial, we will discuss what are generators in Python and how can we create a generator. We will also discuss how it is different from iterators and normal functions. Python Generators

What are the Generators?

Though we can make our own Iterators using a class, __iter__() and __next__() methods, this could be tedious and complex. The generator is a technique used to create user-defined iterators. Basically, generators are the user-defined functions that are used to create user-defined Iterators.

How to create a generator?

As we have mentioned above generators are user-defined functions. Like a user-defined function, we can create a generator using def keyword, but in generators, we use a yield statement instead of a return. If a user-defined function contains a yield statement then it would be considered as a generator. The difference between the return and yield statement is a return statement returns a value from a function and terminates the function whereas the yield returns a value from a function but does not terminate, it pauses the function till the next call.

Generator syntax 

def function_name():
    yield value

Example: 

def gen():
    num=1
   print('This function call has called the first yield statement')
   yield num

    num_2=2
   print('We have used next() function to call the 2nd yield statement')
   yield num_2

    num_3 = 3
   print("This next() call has called the third yield statement")
    yield num_3

a = gen()

# First we will print a and look what is the output
print("-----a has become an iterator----")
print(a)

print("----When we print next(a)----------")
print(next(a))

print("------When we again print next(a)---------")
print(next(a))

print('---Printing next(a) third time-------')
print(next(a))

print('---Printing next(a) forth time and this time it will throw an StopIteration error---------')
print(next((a)))

Output 

----- a has become an iterator----
<generator object gen at 0x02F0F070>
----When we print next(a)----------
This function call has called the first yield statement
1
------When we again print next(a)---------
We have used next() function to call the 2nd yield statement
2
---Printing next(a) third time-------
This next() call has called the third yield statement
3
---Printing next(a) forth time and this time it will throw a StopIteration error---------
Traceback (most recent call last):
StopIteration

Behind the code

In the above code, we create function gen() which has three yield statements which means the generator is able to return 3 values.  When we assign the function gen() to a variable a the variable became an iterator object.

Difference between a normal user-defined function and generator

Normal Function Generator
A normal function can contain only return statement A generator should have a yield statement it could also have a return statement
When we call a function, it starts executing immediately With its yield statement, it returns an Iterator object but it does not start execution immediately
We can not apply next() function on its returned object We can perform next() function on its returned object
Once the value is returned from the function the function gets terminated and give control back to the caller When the yield returns an object, it pauses the function and gives control back to the caller
Every time the function gets call its local variable reset. The local variables of the generator function remain in their state until the next call from the caller.

Python Generator with loops

In the above example, we have used 3 different yield statements to return 3 different values instead of writing a yield statements, again and again, we use loops that can serve the same purpose. We often use loops for the yield statements, this makes our code less complex, and it secures our code from StopIteration error.

Example: 

def gen():
    for num in range(1,4):
        yield num

a = gen()
for c in a:
   print(c)

Output: 

1
2
3

Behind the code

In this example we have used for loop to iterate upon a, here a is an iterator.

Python Generator Expression

Like a lambda anonymous function, we can create anonyms generators, it is also known as generator expression. The generator expression looks like list comprehension the only difference instead of using sq. brackets we use parenthesis. We use a generator instead of list comprehension because it is more memory efficient because it does not store any value.

Example: 

list_comprehension = [i*2 for i in range(4)]
print("The type of list_comprehension is:",type(list_comprehension))

generator = (i*2 for i in range(4))
print("The type of generator is:",type(generator))

Output: 

The type of list_comprehension is: <class 'list'>
The type of generator is: <class 'generator'>

Behind the code

In this example, we have created two objects one is a list and another is a generator. Variable list_comprehension is a list build using python list comprehension and another variable generator is a generator build using generator expression.

Why do we use a generator in Python?

  • It is an easy and clear way of creating our own iterator.
  • It is memory efficient because it does not store any value it just returns values using yield statement.
  • It is used to produce an infinite stream of data.
  • It can be used to perform pipeline operations.
Previous Tutorial Next Tutorial